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ABSTRACT 



Alternate assessments provide a mechanism for students with 
complex disabilities to be included in assessment systems. An integral part 
of maximizing the benefits of assessing students is to include the results of 
alternate assessments in school accountability systems. This report addresses 
policy options for including the results of alternate assessments in school 
accountability systems. Options include: (1) scale the results so that the 

value awarded for achievement levels on the alternate are the same or similar 
to, the value awarded for achievement levels on the general assessment; or (2) 
scale the results so that the achievement levels on the alternate assessment 
are at the lower end of the scale and achievement levels on the general 
assessment are at the upper end of the scale. Examples of Option 1 approaches 
used in several states are provided, including having an alternate assessment 
with different labels and counting rules, having two' alternate assessments 
counted differently, and having an alternate assessment with the same labels 
and same counting. The report concludes by stressing the importance of 
including all students in accountability systems and of monitoring the actual 
effects of various approaches. (CR) 
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Including Alternate Assessment 
Results in Accountability Decisions 



Alternate assessments provide a 
mechanism for students with the 
most complex disabilities to be 
included in assessment systems. 
Like regular assessments, the 
purpose of alternate assessments is 
to provide valid and reliable 
assessment data that accurately 
reflect the state's learning stan- 
dards, and that indicate how a 
school, district, or state is doing in 
terms of overall student perfor- 
mance. 

An integral part of maximizing the 
benefits of assessing students is to 
include the results of those assess- 
ments in school accountability. 

This applies to students with 
disabilities who are assessed 
through alternate assessments. 

The purpose of this Policy Direc- 
tions is to address policy options 
for including the results of alter- 
nate assessments in school ac- 
countability systems. Examples of 
approaches used in several states 
are provided. 




Two policy issues have emerged 
with respect to alternate assess- 
ments - (1) can alternate assess- 
ments be included in school ac- 
countability systems? and (2) what 
is the best way to include alternate 
assessment results in accountabil- 
ity systems? The first question has 
been debated, with arguments for 
including these results focusing on 
the need to count all students and 
arguments against including these 
results focusing on technical 
difficulties in determining how to 
count alternate assessment results. 
These arguments are summarized 
in Table 1 . 

The objective of including alternate 
assessment results in school ac- 
countability is to ensure that the 
students who participate in the 
alternate assessment are included 
in the opportunities to learn that 
are generated by accountability 
systems/and ultimately in the 
improved learning that accompa- 
nies them. Schools can use ac- 
countability information based on 
all students to make policy, admin- 



istrative, and instructional deci- 
sions that improve schooling 
practices so that all students, 
including students in the alternate 
assessment, are successful. 

Starting Point: Good 
Alternate 
^ Assessments 
Alternate assessments are part of 
the standards-based reform initia- 
tive designed to ensure that all 
students attain high standards of 
learning. Thus, they are one part of 
the broad effort to: 

• Define content standards 
describing what all students 
should know and be able to do 

• Define acceptable levels of 
performance 

• Ensure that all students have 
opportunities to learn the 
content 
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Table 1. Pro and Con Arguments for Including Alternate Assessment Results in Accountability 


Pro Arguments 


x . ... r- 'f 

Con Arguments • ;■ : iT , 


• Each student’s assessment score is valued. 

• Scores of alternate assessment can improve. 

• Scores of alternate assessment are not a 
negative factor for school accountability index; 
thus, inclusion is not discouraged. 


• Does not value the “real” level of proficiency that 
must be reached by most students. 

• Could encourage inappropriate placement of 
students in the alternate assessment. 



• Develop technically sound 
assessments to measure student 
performance 

• Develop methods of using the 
assessment results to hold 
schools accountable for stu- 
dents' learning 

Despite being part of the stan- 
dards-based system, alternate 
assessments involve different 
assessment approaches from those 
used in most general assessment 
systems. Thus, questions are raised 
about how results from alternate 
assessments can be combined with 
results from general assessments 
for accountability purposes. 

To address questions about ways 
to include alternate assessment 
results in accountability systems, it 
is essential that the alternate 
assessment be well-developed (see 
Table 2 for some of the characteris- 
tics of well-developed alternate 
assessments). Once this has been 
accomplished, it is much easier to 
identify ways in which the results 
of these assessments can be in- 
cluded in accountability systems. 




Although there are many ways in 
which alternate assessment results 




can be reported (see Bechard in 
Resources), these approaches can 
be merged into two basic options 
for accountability (see Hill in 
Resources). The two options reflect 
different ways of counting alter- 
nate assessment results - either as 
a score that can cover a range of 
values and thus can be improved 
in accountability, or as a score that 
is always at the lowest level. 

These options have been described 
in terms of how the alternate 
assessment results will be scaled 
relative to the results of the general 
assessment (see Hill in Resources): 

► Option 1: Scale the results so 
that the value awarded for 
achievement levels on the 
alternate are the same or 
similar to the value awarded 
for achievement levels on the 
general assessment. 

This approach is based on a belief 
that achievement on the alternate 
assessment is valued just as much 
as achievement on the general 
assessment. The alternate assess- 
ment, like the general assessment, 
must be aligned to the state's 
content standards (usually by 
extending those standards). The 
alternate assessment also has a 



defined alternate achievement 
standard to complement the 
desired achievement standard for 
students in the general assessment. 

In this option, improvements in 
alternate assessment scores are just 
as important as are improvements 
in general assessment scores. A 
student in the alternate assessment 
can achieve proficient status as 
defined for this small population 
of students, just as can a student in 
the general assessment. For this 
approach to work, it is important 
that the population of students for 
whom the alternate assessment is 
appropriate be clearly defined - 
defined in such a way that the 
number of alternate assessment 
students is stable from one year to 
the next. 

The major objection to Option 1 is 
that awarding the same number of 
points or the same proficiency 
label for successful performance on 
the alternate assessment may 
devalue proficient performance on 
the regular assessment. It also 
could promote the inclusion of 
some students in the alternate 
assessment who should be in the 
general assessment. Both of these 
objections can be addressed 
through policy. 
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Table 2. Characteristics of Good Alternate Assessments 



1 . There has been careful stakeholder and policymaker development and definition of desired student 
outcomes for the population, reflecting the best understanding of research and practice. 

2. Assessment methods have been carefully developed, tested, and refined. 

3. Professionally accepted standards are used to score evidence (e.g., adequate training, dual scoring, 
third party tie breakers, reliability tests and rechecks of scorer competence). 

4. An accepted standards-setting process has been used so that results can be included in reporting and 
accountability. 

5. The assessment process is continuously reviewed and improved. 



O Option 2: Scale the results so 
that the achievement levels 
on the alternate assessment 
are at the lower end of the 
scale and achievement levels 
on the general assessment are 
at the upper end of the scale. 
This approach is based on a belief 
that there must be an absolute 
level of achievement that is the 
target for all students, regardless of 
the significance of their disabilities. 
A student taking the alternate 
assessment with this approach can 
never achieve proficient status, but 
instead can only achieve the lowest 
achievement level possible — just 
because the student participates in 
the alternate assessment. 

This approach discourages the 
inappropriate assignment of 
students to the alternate assess- 
ment because the scores a student 
could receive are automatically at 
the lower end of the achievement 
scale. On the other hand, there is 
nothing that can be done to im- 
prove the score of a student in the 
alternate assessment because the 
student will always earn the 
lowest achievement level possible 
regardless of actual improvements 
in achievement. 




The major objection to Option 2 is 
that the presence of students who 
are appropriately assigned to the 
alternate assessment is likely to 
bring down school achievement 
indices. As a result, there is little 
incentive for schools to want these 
students in their buildings, thereby 
discouraging inclusive educational 
settings for alternate assessment 
students. 




Comparison of 



The selection of one accountability 
option over another is a policy 
decision - one that should be 
reached through stakeholder 
involvement. The decision also 
should be based on evidence from 
states that have adopted one 
approach or another, and by 
consideration of the results of 
simulation research. 



The effects of adopting Option 1 
have been demonstrated in simula- 
tion research conducted by Richard 
Hill from the Center for the Im- 
provement of Educational Assess- 
ment (see Resources). Three con- 
clusions were supported when he 
used state data in simulated 
formulas for accountability: 






1. The impact of including alter- 
nate assessment scores on 
school gains is trivial if the 
number of alternate assessment 
participants remains fairly 
constant from year to year. 

2. Making gains on the alternate 
assessment that are comparable 
to gains on the general assess- 
ment introduced little addi- 
tional measurement error. 

3. Including alternate assessment 
results in accountability ap- 
pears to lead to better outcomes 
for the students who participate 
in the alternate assessment. 

If a school or district is looking for 
gains in achievement for all stu- 
dents, the school should be entitled 
to rewards for gains in the alter- 
nate that are equivalent to rewards 
for gains in general assessments. 

Hill concludes that states should 
address several essential questions 
as they consider options for includ- 
ing alternate assessment scores in 
accountability: What is fair? What 
will encourage the greatest im- 
provement for every student? 

What seems reasonable? Thus far, 
states have answered these ques- 
tions in different ways. 
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► Examples of State 
Approaches to 
Accountability 
Option 1 

To some extent, the application of 
the accountability options in states 
will reflect the differences in the 
assessment systems on which 
accountability is based. The three 
approaches described here begin 
with the assumption that all 
students count; the question of 
whether results can be included 
has already been answered: yes 
they can. The approaches differ in 
the specific ways that results are 
included. 

Approach 1: Alternate 
Assessment with Different 
Labels and Counting Rules 

In this approach, achievement 
levels for the general assessment 
and alternate assessment are 
different, as are the descriptors of 
the levels. For example, one state 
refers to the four achievement 
levels of the general assessment as 
novice, partially proficient, proficient, 
and advanced. In contrast, for the 
alternate assessment, there are 
three achievement levels, with the 
labels beginning, partially skilled, 
and skilled. 

In this first approach to including 
alternate assessment results in 
accountability, a set of decision 
rules is used to determine when 
the alternate assessment results are 
considered. The decision rules 
reflect the importance placed on 
appropriate participation in assess- 
ments. Thus, alternate assessment 
results are included when there is 
no evidence of sufficient participa- 
tion in the general assessment, and 
when adequate improvement in 



general assessment scores relative 
to a long-range target is not evident. 

In this approach, any school that 
has a general assessment participa- 
tion rate that is less than 95% 
automatically goes into school 
improvement. For those schools 
without adequate gains in general 
assessment performance, other 
factors are considered as well (for 
example, participation rates for the 
alternate assessment, grade 1 and 2 
assessment results, progress of 
Title I students, reductions in 
percentage of students in the novice 
level). 

Schools gain or lose points based 
on a combination of participation 
and progress on the alternate 
assessment. Specific points added 
or subtracted from school scores 
are as follows: 

• Add 4 points - progress in 
average alternate assessment 
score and/or 99% participation 
rate in the general assessment 

• Add 2 points - no progress in 
alternate assessment score and/ 
or 99% participation rate in the 
general assessment 

• 0 points - decline in average 
alternate assessment scores 
with a 98% participation rate in 
the general assessment 

• Subtract 2 points - participation 
rate in general assessment was 
97% 

• Subtract 4 points - participation 
rate in general assessment was 
less than 97% 

Obviously, the specifics of the way 
in which the alternate assessment 
factors into the accountability 
system can be altered, but the key 
element of this approach is that the 
alternate assessment is counted in 
some way. 



Advantages. An emphasis on 
getting all students into the system 
is likely to result in high participa- 
tion rates. Schools can improve 
their status through full participa- 
tion and through improved alter- 
nate assessment performance. 

Disadvantages. Different labels for 
achievement levels and use of 
alternate assessment results only 
when general assessment partici- 
pation and performance is not 
considered adequate essentially 
gives less weight to the alternate 
assessment. For schools making 
adequate gains with their general 
assessment students but inad- 
equate gains (or no gains) with 
their alternate assessment stu- 
dents, there is no recognition of the 
need to improve programs for 
those students ’with the most 
significant disabilities. 

^•Approach 2: Two Alternate 
Assessments Counted Dif- 
ferently 

In this approach there are two 
alternate assessments, one of 
which is intended for students 
with significant disabilities (gener- 
ally an alternate portfolio), and the 
other of which is for students 
"who may not be able to take the 
general assessment but who are 
not eligible for the alternate portfo- 
lio" (generally some type of lower 
grade-level assessment or an 
academic inventory). Each alter- 
nate assessment is included in 
accountability, but in different 
ways. 

The academic assessment, which is 
most like a general assessment, 
expands the academic achievement 
levels of the general assessment. A 
common approach is to add levels 
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to each of the existing achievement 
levels. For example, there may be 
Novice 1, 2, 3, and 4; Apprentice \, 2, 
3, and 4; Proficient \, 2, 3, and 4; 
and Distinguished 1, 2, 3, and 4. 

This essentially provides more 
steps for demonstrating progress. 
These scores can be included in 
accountability in the same way 
that general assessment scores are 
- generally one number for each 
content area (reading, math, and 
others). 

The alternate portfolio is included 
in this accountability approach in a 
different way from the alternate 
academic inventory. While the 
same labels are given to the alter- 
nate portfolio achievement levels 
( novice , apprentice, proficient, distin- 
guished) as are given to the general 
assessment achievement levels 
(without the extra levels 1, 2, 3, 
and 4 given to the academic alter- 
nate), each student receives only 
one score rather than the multiple 
scores in the general assessment 
and the alternate academic inven- 
tory. Thus, the general assessment 
counts several times, as does the 
alternate academic inventory, 
while the alternate portfolio counts 
just once. 

Advantages. All students receive 
scores and all scores count. The use 
of the same labels conveys a 
message that students with the 
most significant disabilities can 
become proficient and distin- 
guished, just as other students can. 

Disadvantages. Scores of students in 
the alternate portfolio count less 
than the scores of other students. 
This approach essentially averages 
scores rather than averaging 
students, and some students count 
more than others. Schools may 
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earn awards without any impact 
on students in the alternate portfo- 
lio. This may lead to less concern 
about improving programs and 
instruction for these students. 

[> Approach 3: Alternate 
Assessment with the Same 
Labels and Same Counting 

This approach also uses the same 
labels for the achievement levels 
assigned to the general assessment 
and the alternate portfolio assess- 
ment - novice, apprentice, proficient, 
and distinguished. The belief that 
students who demonstrate profi- 
cient performance within the 
structure of the alternate assess- 
ment should make the same 
contribution to the school account- 
ability index is carried into the 
numbers of the accountability 
system. The same point values 
employed for the general assess- 
ment are used for the alternate 
portfolio assessment. In addition, 
students in the alternate assess- 
ment are required to demonstrate 
achievement within the same 
multiple content areas as are 
students in the general assessment, 
and scores are assigned accord- 
ingly for each content area. 

To further support the commit- 
ment to inclusion of all students in 
the accountability system, this 
approach generally has a couple of 
additional aspects. One is to assign 
the lowest score possible to any 
student who is not in either the 
general or alternate assessment. To 
guard against inappropriately 
including students in the alternate 
assessment, an audit point is 
identified (generally 1-2% of all 
students) to indicate that the level 
of participation in the alternate 
assessment needs to the checked 
because it might be too high. 



Advantages. Scores of students in 
the alternate assessment count just 
the same as scores of students in 
the general assessment. This 
approach essentially averages 
students, rather than test scores, so 
that each student receives equal 
weight in the school accountability 
index. With the scores of all stu- 
dents counting (even those not in 
either assessment), participation is 
encouraged. Further, schools are 
unlikely to earn awards without 
having an impact on students in 
the alternate portfolio. This may 
lead to more concern about im- 
proving programs and instruction 
for these students. 

Disadvantages. Without certain 
additions, this approach could 
result in students being inappro- 
priately placed in the alternate 
assessment. Additions, such as 
assigning the lowest score possible 
to non-participating students and 
monitoring participation rates 
through an audit point, can result 
in an approach with no obvious 
disadvantages. 



The imperative to include all 
students in school accountability 
systems and the decisions that are 
made based on these systems is 
clear. Yet, there are multiple ways 
in which scores can be included, 
and the effects of each approach 
needs to be carefully considered. 
The two most basic options (1) to 
count achievement on the alternate 
assessment comparable to achieve- 
ment on the general assessment, or 
(2) to count achievement on the 
alternate assessment as automati- 
cally lower than achievement on 
the general assessment, is the 
starting point for conversations 
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about which approach to take for 
accountability. 

There are many approaches that 
can be taken within each option. In 
this document we have high- 
lighted those approaches that are 
consistent with the first option 
(where alternate assessment scores 
have the same or similar value as 
general assessment scores), the 
option that has been supported by 
simulation research on the techni- 
cal, motivational, and policy effects 
of different approaches. As imple- 
mentation of different approaches 
proceeds, it will be important to 
monitor the actual effects of vari- 
ous approaches. 
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